1. Field of the Invention
The present invention relates to an image processing apparatus, and specifically, relates to an image processing apparatus, an imaging apparatus, and a solid-state imaging device, which subject image data to an image process, and a processing method according thereto, and a program causing a computer to execute the method thereof.
2. Description of the Related Art
In recent years, CCD (Charge Coupled Device) sensors, CMOS (Complementary Metal Oxide Semiconductor) sensors, and so forth have come into widespread use as solid-state imaging devices which image a subject to generate image data. The dynamic range as to incident light of these solid-state imaging devices (i.e., the luminance range from a level wherein a signal is inextricable due to being lost in floor noise, to a saturation level) is improving steadily by progress of semiconductor technology. However, in reality, it is not so uncommon for incident light exceeding a predetermined dynamic range to occur when using solid-state imaging devices. Therefore, technology used for expanding a dynamic range is being actively studied.
As a technique used for obtaining a high dynamic range image, for example, a technique has been proposed wherein multiple images, imaged using different exposures, are synthesized into one image. However, with such a technique, exposure time for imaging of multiple images increases, and accordingly, such a technique has a problem in that moving subject blurring or camera shaking readily occurs. As a technique to solve this problem, for example, imaging technology has been proposed wherein multiple pixels having different sensitivity are disposed on the imaging surface of a single image sensor, thereby executing multistep exposure at once to generate a high dynamic range image (e.g., see International Publication WO 2002/056603 pamphlet (FIG. 1)). Imaging having a different sensitivity can be performed simultaneously by employing this imaging technology, whereby the whole exposure period can be relatively shortened. However, this imaging technology is the same as other existing imaging technology in that the sensitivity itself of the solid-state imaging device is not improved. Therefore, in the event that there is a low luminance subject, the exposure period increases in accordance with this low luminance subject, and occurrence of camera shaking is not reduced.
On the other hand, as for technology used for correcting an image of which the camera shaking has occurred, for example, technology has been proposed wherein a short exposure image (short-time-exposure image) and a long exposure image (long-time-exposure image) are employed to correct camera shaking that has occurred during the long-time-exposure image (e.g., see L. Yuan, J. Sun, L. Quan, H. Y. Shum: “Image Deblurring with Blurred/Noisy Image Pairs”, Proceedings of ACM SIGGRAPH 2007, Article 1, 2007). With this camera shaking correction technology, a short-time-exposure image is employed to estimate the camera shaking function PSF (Point Spread Function) of the long-time-exposure image, and subject the long-time-exposure image to reverse correction thereof to correct camera shaking that has occurred upon the long-time-exposure image. That is to say, the short-time-exposure image is employed as reference data to estimate the camera shaking PSF of the long-time-exposure image.
Also, for example, there has been proposed camera shaking correction technology that employs an imaging apparatus capable of measuring operations of the imaging apparatus itself. For example, there has been proposed an imaging apparatus in which a high-resolution image sensor which executes imaging by long exposure, and a low-resolution image sensor which measures camera shaking by short consecutive exposure are combined (e.g., see M. Ben-Ezra, S. K. Nayar: “Motion Deblurring Using Hybrid Imaging”, Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition, Vol. I, pp 657-664, 2003). This camera shaking correction technology is camera shaking correction technology wherein the camera shaking PSF of a long-time-exposure image is estimated by measurement using a short-time-exposure image, and camera shaking that has occurred on the long-time-exposure image is corrected by subjecting this image to reverse correction thereof. That is to say, the short-time-exposure image generated by the low-resolution image sensor is used to estimate the camera shaking PSF of the long-time-exposure image.